CSDV3017  ·  DEVOPS  ·  SCHOOL OF COMPUTER SCIENCE, UPES

Intro to Testing, Verification vs Validation, Types of Testing

Lecture 15 — kicking off Unit VI by exploring the critical role of software testing in DevOps, understanding the difference between Verification and Validation, and breaking down testing types from White-box to Automated.

InstructorDr. Mohsin Furkh Dar
SessionWeek 5 · Tue, 07 Jul 2026
Time14:00 – 15:00
UnitUnit VI
I
II
III
IV
V
VI
VII
WELCOME TO UNIT VI

Agenda: Software Testing

Unit V Recap

Tools & Hands-on

  • Docker containerization vs VMs
  • Kubernetes orchestration architecture
  • Puppet vs Ansible for Config Management
  • Writing CI workflows (GitHub Actions) & IaC (Terraform)
Lecture 15 · Today

Testing Fundamentals

  • Why do we test in DevOps? (Shift-Left Testing)
  • Verification vs. Validation
  • White-box vs. Black-box testing
  • Manual vs. Automated testing
  • The Testing Pyramid
THE COST OF BUGS

Why testing matters in DevOps

"If you automate a mess, you get an automated mess." — DevOps Proverb

The CI/CD Dependency

CI/CD is only as good as its tests

In a Continuous Deployment pipeline, there is no manual gatekeeper. If code merges to main, it goes to production. The only thing stopping a critical bug from destroying the production database is the automated test suite. High-velocity deployments require absolute trust in your testing.

Traditional IT

Testing at the end

QA team tests the software right before release. Bugs are expensive to fix because code is already written.

DevOps Approach

Shift-Left Testing

Move testing to the "left" (earlier) in the lifecycle. Developers write and run tests as they code.

The Goal

Fast Feedback

Developers should know within 5 minutes of a commit if they broke something, not 5 weeks later.

V & V

Verification vs. Validation

Verification

"Are we building the product right?"

  • Checks if the software conforms to its technical specifications and design.
  • Focuses on the internal architecture, code quality, and logic.
  • Activities: Code reviews, static analysis, unit testing, debugging.
  • Outcome: A product without internal defects.
Validation

"Are we building the right product?"

  • Checks if the software meets the actual needs and expectations of the user.
  • Focuses on the external behavior, usability, and business logic.
  • Activities: User acceptance testing (UAT), A/B testing, beta testing.
  • Outcome: A product that solves the user's problem.

Analogy: Verification checks if the car's engine works perfectly. Validation checks if a car is actually what the customer wanted, or if they needed a boat.

TESTING APPROACHES

White-box vs. Black-box

Feature White-box Testing (Clear Box) Black-box Testing (Opaque Box)
Definition Testing internal structures or workings of an application. Testing functionality without looking at internal structures.
Knowledge Required Tester must know programming and the internal codebase. Tester only needs to know the expected inputs and outputs.
Performed By Usually done by Developers. Usually done by QA Engineers / Users.
Focus Code coverage, branches, paths, logic flow. UI, usability, business rules, edge case inputs.
Examples Unit tests (JUnit, Jest), Static Analysis. E2E tests (Selenium), User Acceptance Testing.
THE MIDDLE GROUND

Gray-box Testing

Definition

What is Gray-box testing?

A combination of White-box and Black-box testing. The tester has partial knowledge of the internal structure (like database schemas or API architectures) and uses that knowledge to design better black-box tests.

Example Scenario

Testing an API

You test the application via its UI (Black-box), but because you know the backend database structure (White-box), you intentionally input data that you know might break a specific SQL query (SQL Injection testing).

Why use it?

Best of both worlds

  • Uncovers deep-level logic errors that purely black-box testers miss.
  • Tests from a user's perspective, unlike pure white-box testing.
  • Highly effective for Integration Testing in microservices.
EXECUTION METHODS

Manual vs. Automated Testing

Manual Testing

Human Intelligence

  • A human clicks through the application, checking for issues.
  • Pros: Finds visual issues, UX flaws, and unexpected edge cases ("exploratory testing").
  • Cons: Slow, expensive, prone to human error, cannot be run on every commit.
  • Best for: Usability, ad-hoc testing, UAT.
Automated Testing

Machine Speed

  • Scripts run the tests and compare actual outcomes to expected outcomes.
  • Pros: Extremely fast, repeatable, runs in CI/CD pipelines, catches regressions instantly.
  • Cons: High initial setup cost, scripts need maintenance, cannot test "feel".
  • Best for: Regression testing, load testing, performance.

The DevOps Mandate: Automate everything you can (regression, unit, integration), so humans can spend their manual testing time on creative, exploratory testing where human intelligence is actually needed.

TESTING STRATEGY

The Testing Pyramid

Introduced by Mike Cohn, the pyramid visualizes how a healthy automated testing portfolio should be structured.

Base (70%)

Unit Tests

  • Test individual functions/classes.
  • Very fast (milliseconds).
  • Cheap to write.
  • Highly isolated (mocked DBs).
  • Tools: JUnit, Jest, PyTest.
Middle (20%)

Integration Tests

  • Test how components talk to each other (App + Database).
  • Slower (seconds).
  • Catches API/contract issues.
  • Tools: Postman, RestAssured.
Top (10%)

End-to-End (E2E)

  • Test the whole app through the UI.
  • Very slow (minutes).
  • Expensive and brittle (flaky).
  • Tools: Selenium, Cypress, Playwright.

Anti-Pattern: The Ice Cream Cone. Many legacy companies have an inverted pyramid — tons of slow, manual E2E tests, and very few fast unit tests. This kills CI/CD velocity.

BEYOND FUNCTIONALITY

Non-Functional Testing

Test Type Objective Tools Example
Performance Testing Checks how fast the system responds under normal workload. JMeter, Gatling
Load Testing Checks system behavior under expected peak load (e.g., Black Friday traffic). Locust, k6
Stress Testing Pushes the system beyond limits to see how it fails and recovers. Chaos Monkey, Gremlin
Security Testing (SAST/DAST) Finds vulnerabilities, SQL injections, and weak encryption. SonarQube, OWASP ZAP
Usability Testing Evaluates how user-friendly and intuitive the interface is. Hotjar, UserTesting
WRAP-UP

Summary & what's next

Lecture 15 · Key Takeaways

What you should remember

  • Shift-Left: Testing earlier in the dev cycle to catch bugs when they are cheap to fix.
  • Verification vs Validation: "Building it right" (Dev focus) vs "Building the right thing" (User focus).
  • White-box vs Black-box: Internal logic vs External behavior.
  • Manual vs Automated: Human exploratory testing vs High-speed repeatable scripts.
  • Testing Pyramid: Lots of fast Unit tests, some Integration tests, few slow E2E tests.
Next Lecture · Lecture 16

Mon, 13 Jul 2026 · 14:00–15:00 · Unit VI

Software build process; Test case writing; Automation testing tools in-depth; Manual vs Automated Deployment; IBM Case Study part 2.

Prep for next class

Automation tools

We will look closely at tools like Selenium and Cypress. Think about a repetitive task on a website you'd like to automate.

CSDV3017 · DEVOPS
SHEET 01/10